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that  an  object  or  event  A  belongs  to  a  class  or  process  B.  (ii)  An  assessment  of 
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development,  (iii)  An  adjustment  from  a  starting  point,  which  is  usually  employed 
in  numerical  prediction  when  a  relevant  value  is  available.  These  heuristics  are 
highly  economical  and  usually  effective,  but  they  lead  to  systematic  and  predicta¬ 
ble  errors.  A  better  understanding  of  these  heuristics  and  of  the  biases  to  which 
they  lead  could  improve  judgments  and  decisions  in  situations  of  uncertainty. 
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JUDGMENT  UNDER  UNCERTAINTY:  HEURISTICS  AND  BIASES 
Amos  Tversky  and  Daniel  Kahneman 
Hie  Hebrew  University 
Jerusalem 


Most  important  decisions  are  based  on  beliefs  concerning  the 
likelihood  of  uncertain  events  such  as  the  outcome  of  an  election, 
the  guilt  of  a  defendant,  or  the  future  value  of  the  dollar.  These 
beliefs  are  usually  expressed  in  statements  such  as  "I  think  that...", 
"chances  are.,.",  "It  is  unlikely  that...",  etc.  Occasionally,  beliefs 
concerning  uncertain  events  are  expressed  in  a  numerical  form  as  odds 
or  subjective  probabilities.  What  determines  such  beliefs?  How  do  people 
assess  the  likelihood  of  an  uncertain  event  or  the  value  of  an  uncertain 
quantity?  The  theme  of  the  present  paper  is  that  people  rely  on  a  limited 
number  of  heuristic  principles  by  which  they  reduce  the  complex  tasks  of 
assessing  likelihoods  and  predicting  values  to  simpler  judgmental  opera¬ 
tions.  In  general,  these  heuristics  are  quite  useful,  but  sometimes  they 
lead  to  severe  and  systematic  errors. 

The  intuitive  assessment  of  probability  resembles  the  assessment  of 
perceptual  quantities  such  as  distance  or  size.  These  judgments  are  all 
based  on  data  of  limited  validity,  which  is  processed  according  to  heuristic 
rules.  For  example,  the  apparent  distance  of  an  object  is  determined  in  part 
by  its  clarity.  The  more  sharply  the  object  is  seen,  the  closer  it  appears 
to  be.  This  rule  has  some  validity,  because  in  any  given  scene  the  more 
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ohjccrs  .1 re  seen  less  sharply  than  nearer  objects.  However, 
the  reliance  „„  this  ru.c  leads  to  systematic  errors  in  the  eslima- 
of  distance.  Speci 1 ically ,  distances  are  often  overestimated 
visibility  is  poor  because  the  contours  of  objects  are  blurred, 
othei  hand,  distances  are  often  underestimated  when  visibility 
is  good  because  the  objects  are  sharply  seen.  Three  features  of  this 
example  are  worth  noting,  (c)  People  are  not  generally  aware  of  the 
rules  that  govern  their  impressions;  they  are  normally  ignorant  of 
the  important  role  of  blur  in  the  perception  of  distance,  (iij  People 
cannot  deliberately  control  their  perceptual  impressions:  a  sharply 
seen  hilltop  looks  near  even  if  „„e  has  learned  of  the  effect  of  clarity 
on  the  perception  of  distance.  <i,„  „  is  possible  to  learn  to  recognise 
the  situations  in  which  impressions  are  likely  to  he  biased.and  to  deli- 
berate, y  make  appropriate  corrections,  in  making  a  decision  to  climb  a 
hill,  for  example,  one  should  consider  the  possibility  that  the  summit  is 
further  than  it  looks  if  the  day  is  particularly  clear. 

A  similar  analysis  applies  to  the  assessment  of  likelihoods  and  to 
the  prediction  of  values.  As  in  the  perceptual  example,  people  apply  heuris- 
ttc  rules  to  their  fallible  impressions.  Here  too,  people  are  rarely  „.are  „f 
the  basis  of  their  impressions ,.  and  t  oy  have  little  deliberate  centre,  over 
the  processes  by  which  these  impressions  are  formed.  However,  they  can  iearn 

to  identify  the  heuristic  processes  that  •  .  . 

P  ocesses  that  determine  their  impressions,  and 

tc  make  appropriate  allowances  for  the  hia^«.  *  -  . 

lases  to  w.uch  they  are  liable.  The 

following  sections  describe  three  heuristics  that  a 
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3- 


assess  likelihoods  and  to  predict  v.,,„„s;  ...numerate  systematic  biases 

to  which  these  hcnrist.cs  lead;  and  discuss  the  applied  and  theoretical 
implications  ol  ibis  researcli. 

REPRIiSliNTAT  I  VHNIiSS 

Many  oi  the  probabilistic  questions  with  which  people  are  concerned 

belong  to  one  ol  the  following  types:  Want  is  the  probability  that  an 

object  A  belongs  to  a  class  B?  What  is  the  probability  that  event  A 

originates  from  process  15?  What  is  the  probability  that  process  A  will 

generate  an  event  B?  In  answering  such  questions  people  typically  rely 

on  the  representativeness  heuristic,  in  which  probabilities  are  evaluated 

by  the  degree  to  which  A  is  representative  of  B,  i.e.,  by  the  degree  of 

similarity  between  them.  When  A  and  B  are  very  similar,  e.g.,  when  the 

outcome  in  question  is  highly  representative  of  the  process  from  which 

it  rriginates,  then  its  probability  is  judged  to  be  high.  If  the  outcome 

is  not  representative  of  the  generating  process,  probability  is  judged  to 
be  low. 

For  an  illustration  of  judgment  by  representativeness,  consider  an 
individual,  Mr.  X,  who  has  been  described  a.  "meticulous,  introverted, meek, 
solemn  ,  and  the  following  set  of  occupational  roles:  farmer,  salesman,  pilot, 
librarian,  physician.  How  do  people  evaluate  the  likelihood  that  Mr.  X  is 
g  g  in  each  of  these  occupations,  and  how  do  they  order  the  occupations 
terms  of  likelihood?  In  the  representativeness  heuristic,  one  assesses 
the  similarity  of  Mr.  X  to  the  stereotype  of  each  occupational  role,  and 
orders  the  occupations  by  the  degree  to  which  Mr.  X  is  representative  of 
these  stereotypes.  Research  with  problems  of  this  type  has  shown  that  people 


in  fact  order  the  Occupations  by  likelihood  and  by  similarity  in 
exactly  the  same  way  (l).  As  will  be  shown  below,  this  approach  to 
the  judgment  of  likelihood  leads  to  serious  biases,  because  several 
of  the  factors  that  should  be  considered  in  assessing  likelihood  play 
no  role  in  judgments  of  similarity. 


1-  Insensitivity  to  prior  probability  of  outcomes. 

One  of  the  factors  that  have  a  major  effect  on  probability 
but  has  no  effect  on  representativeness  is  the  prior  probability, 
or  base-rate  frequency,  of  the  outcomes.  In  the  case  of  Mr.  X. 
for  example,  the  fact  that  there  are  many  more  farmers  than  libra¬ 
rians  in  the  population  should  enter  into  any  reasonable  estimate  of  the 
probability  that  Mr.  X  is  a  librarian  rather  than  a  farmer.  Considerations 
of  base-rate  frequency,  however,  do  not  affect  the  similarity  of  Mr.  X.  to 
the  stereotypes  of  librarians  and  farmers.  If  people  evaluate  probability 
by  representativeness,  therefore,  prior  probabilities  will  be  neglected. 

hypothesis  was  tested  in  an  experiment  where  prior  probabilities  were 
explicitly  manipulated  (1).  Subjects  were  shown  brief  personality  descriptions 
several  individuals,  allegedly  sampled  at  random  from  a  *_roup  of  100  pro¬ 
fessionals  -  engineers  and  lawyers.  The  subjects  were  asked  to  assess,  for 
each  description,  the  probability  that  it  belonged  to  an  engineer  rather 
than  to  a  lawyer.  In  one  experimental  condition,  the  subjects  were  told 
that  the  group  from  which  the  descriptions  had  been  drawn  consisted  of  70 
engineers  and  30  lawyers.  In  another  condition,  subjects  were  told  that  the 
group  consisted  of  30  engineers  and  70  lawyers.  The  odds  that  any  particular 
description  belongs  to  an  engineer  rather  than  to  a  lawyer  should  be  higher 
condition,  where  there  is  a  majority  of  engineers,  than  in  the 
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second  condition,  where  there  is  a  majority  of  lawyers.  Specifically, 
it  can  be  shown  by  applying  Bayes'  rule  that  the  ratio  of  these  odds 
should  be  (.7/. 3) 2  =-  5.44  for  each  description.  In  sharp  contrast  to 
Bayes  rule,  the  subjects  in  the  two  conditions  produced  essentially 
the  same  probability  judgments.  Apparently,  subjects  evaluated  the 
likelihood  that  a  particular  description  belonged  to  an  engineer 
rather  than  to  a  lawyer  by  the  degree  to  which  this  description  was 
representative  of  the  respective  stereotypes,  with  little  or  no  regard 
for  the  prior  probabilities  of  the  two  outcomes. 

The  subjects  correctly  utilized  prior  probabilities  when  they  had 
no  other  information.  In  the  absence  of  a  personality  sketch  they  judged 
the  probability  that  an  unknown  individual  is  an  engineer  to  be  .7  and 
.3  respectively,  in  the  two  base-rate  conditions.  However,  prior  proba¬ 
bilities  were  effectively  ignored  when  a  description  was  introduced,  even 
when  this  description  was  totally  uninformative.  The  responses  to  the 
following  description  illustrate  this  phenomenon: 

Dick  is  a  30-year  old  man.  He  is  married 
with  no  children.  A  man  of  high  ability 
and  high  motivation,  he  promises  to  be 
quite  successful  in  his  field.  He  is  well 
liked  by  his  colleagues. 

This  description  was  intended  to  convey  no  information  relevant  to  the 
question  of  whether  Dick  is  an  engineer  or  a  lawyer.  Consequently,  the 
probability  that  Dick  is  an  engineer  should  equal  the  proportion  of  engineer 
in  the  group,  as  if  no  description  had  been  given.  The  subjects,  however, 
judged  the  probability  of  Dick  being  an  engineer  to  be  .5  regardless  of 
whether  the  stated  proportion  of  engineers  in  the  group  was  .7  or  .3. 
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Evidently,  people  respond  differently  when  given  no  evidence  and  when 
given  worthless  evidence  (1).  When  rio  specific  evidence  is  given  -  prior 
probabilities  are  properly  utilized;  when  worthless  evidence  is  given  - 
prior  probabilities  are  ignored. 

2.  Insensitivity  to  s amp  1 o  size. 

To  evaluate  the  probability  of  obtaining  a  particular  result  in  a 
sample  drawn  from  a  specified  population,  people  typically  apply  the 
representativeness  heuristic.  That  is,  they  assess  the  likelihood  of  a 
sample  result  (e.g.,  that  the  average  height  in  a  random  sample  of  ten 
men  will  be  6'0")  by  the  similarity  of  this  result  to  the  corresponding 
parameter  (i.e.,  to  the  average  height  in  the  population  of  men).  The 
similarity  of  a  sample  statistic  to  a  population  parameter  is  unaffected 
by  the  size  of  the  sample.  Consequently,  if  probabilities  are  assessed 
by  representativeness,  then  the  judged  probability  of  a  sample  statistic 
will  be  essentially  independent  of  sample  size.  Indeed,  when  subjects 
assessed  the  distributions  of  average  height  for  samples  of  various  sizes, 
they  produced  identica  distributions.  For  example,  the  probability  of 
obtaining  an  average  height  greater  than  6'0"  was  assigned  the  same  value 
for  samples  of  1000,  100,  and  10  men  (2).  Moreover,  subjects  failed  to 
appreciate  the  role  of  sample  size  even  when  it  was  emphasized  in  the  formu¬ 
lation  of  the  problem.  Consider  the  following  question: 

A  certain  town  is  served  by  two  hospitals. 

In  the  larger  hospital  about  45  babies  are 
born  each  day,  and  in  the  smaller  hospital 
about  15  babies  are  born  each  day.  As  you 
know,  about  50%  of  all  babies  are  boys.  The 
exact  percentage  of  baby  boys,  however,  varies 
from  day  to  day.  Sometimes  it  may  be  higher  than 
50%,  sometimes  lower. 
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For  a  pariod  of  one  year,  each  hospital  recorded 
the  days  on  which  more  than  60%  of  the  babies  bom 
were  boys.  Which  hospital  do  you  think  recorded  more 
such  days? 

-  The  larger  hospital?  (21) 

-  The  smaller  hospital?  (21) 

-  About  the  same?  (i.e.,  within  5%  of  each  other)  15*1 

The  values  in  parenthesis  are  the  number  of  undergraduate  students  who  chose 
each  of  the  three  answers. 

Most  subjects  judged  the  probability  of  obtaining  more  than  60%  boys 
to  be  the  same  in  the  small  and  in  the  large  hospital,  presumably  because 
these  events  are  described  by  the  same  statistic  and  are  therefore  equally 
representative  of  the  general  population.  In  contrast,  sampling  theory  en¬ 
tails  that  the  expected  number  of  days  on  which  more  than  60%  of  the  babies 
are  boys  is  much  greater  in  the  small  hospital  than  in  the  large  one,  because 
a  large  sample  is  less  likely  to  stray  from  50%.  This  fundamental  notion  of 
statistics  is  evidently  not  part  of  people's  repertoire  of  intuitions. 

A  similar  insensitivity  to  sample  size  has  been  reported  in  judgments 
of  posterior  probability,  i.e.,  of  the  probability  that  a  sample  has  been 
drawn  from  one  population  rather  than  from  another.  Consider  the  following 
example: 

Imagine  an  urn  filled  with  balls,  of  which  2/3  are 
of  one  color  and  1/3  of  another.  One  individual  has 
drawn  5  balls  from  the  urn,  and  found  that  4  were  red 
and  1  was  white.  Another  individual  has  drawn  20  balls 
and  found  that  12  were  red  and  8  were  white.  Which  of 
the  two  individuals  should  feel  more  confident  that  the 
urn  contains  2/3  red  balls  and  1/3  white  balls,  rather 
than  the  opposite?  What  odds  should  each  individual 
give? 
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In  this  problem,  the  correct  posterior  odds  are  8  to  1  for  the 
1:1  sample  and  16  to  I  for  the  12:8  sample,  assuming  equal  prior 
probabilities.  However,  most  people  feel  that  the  first  sample  pro¬ 
vides  much  stronger  evidence  for  the  hypothesis  that  the  urn  is  pre 
dominantly  red,  because  the  proportion  ot  red  balls  is  larger  in  the 
first  than  in  Hie  m.  oml  sample,.  Here  again,  intuitive  judgments  are 
dominated  by  the  sample  proportion  and  are  essentially  unaffected  by 
the  size  of  the  sample,  which  plays  a  crucial  role  in  the  determination 
of  the  actual  posterior  odds  (2).  In  addition,  intuitive  estimates  of 
posterior  odds  are  far  less  extreme  than  the  correct  values.  The  under¬ 
estimation  of  the  impact  of  evidence  has  been  observed  repeatedly  in 
problems  of  this  type  (3,4).  It  has  been  labeled  "conservatism." 

5.  Misconceptions  of  Chance. 

1  cop  1 e  expect  that  a  sequence  of  events  generated  by  a  random  pro- 
cocj  will  represent  the  essential  characteristics  of  that  process  even 
vhen  the  sequence  is  short.  In  considering  tosses  of  a  coin,  for  example, 
people  regard  the  sequence  HTHTTH  to  be  more  likely  than  the  sequence 
HHHTTT,  which  does  not  appear  random,  and  also  more  likely  than  the  sequence 
HHHHTH,  which  does  represent  the  fairness  of  the  coin  (2).  Thus,  people  ex¬ 
pect  that  the  essential  characteristics  of  the  process  will  be  represented, 
not  only  globally  in  the  entire  sequence,  but  also  locally  in  each  of  its 
parts.  A  locally  representative  sequence,  however,  deviates  systematically 
from  chance  expectation:  it  contains  too  many  alternations  and  too  few  runs. 
Another  consequence  of  the  same  belief  is  the  well-known  gambler's  fallacy. 

* 
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Atter  observing  ;i  long  run  of  _rod  on  the  roulette  wheel,  lor  ex;*;upl'\ 
most  people  erroneously  believe  that  black  is  now  due,  presumably  be¬ 
cause  the  occurrence  of  b lack  will  result  in  a  more  representative 
sequence  than  the  occurrence  of  an  additional  red.  In  general,  chance 
is  commonly  viewed  as  a  self-correcting  process  where  a  deviation  m  no¬ 
di  rect ion  induces  a  deviation  in  the  opposite  direction  to  restore  the 
equilibrium.  In  fact,  deviations  are  not  "corrected"  as  a  chance  process 
unfolds,  they  are  merely  diluted. 

Misconceptions  of  chance  are  not  limited  to  naive  subjects.  A 
study  of  the  statistical  intuitions  of  experienced  research  psycholo¬ 
gists  (5)  revealed  a  lingering  belief  in  what  may  be  called  the  "law  of 
small  numbers"  according  to  which  even  small  samples  are  highly  represen¬ 
tative  of  the  populations  from  which  they  are  drawn.  The  responses  of 
these  investigators  reflected  the  expectation  that  a  valid  hypothesis 
about  a  population  will  be  represented  by  a  statistically  significant 
result  in  a  sample  -  with  little  regard  for  its  size.  As  a  consequence, 
the  researchers  put  too  much  faith  in  the  results  of  small  samples,  and 
grossly  overestimated  the  replicability  of  such  results.  This  bias  has 
pernicious  consequences  for  the  conduct  of  research:  it  leads  to  over¬ 
interpretation  of  findings  and  to  the  choice  of  inadequate  sample  sizes. 


^ •  jnsens^tivity  to  predictive  accuracy. 

People  are  sometimes  called  upon  to  make  numerical  predictions,  e.g., 
of  the  future  value  of  a  stock,  the  demand  for  a  commodity,  or  the  outcome 
of  a  football  game.  Such  predictions  are  often  made  by  representativeness. 
For  example,  suppose  one  is  given  a  description  of  a  company,  and  is  asked 


- - 
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to  predict  its  future  profit.  If  the  description  of  the  company  is 
very  favorable,  a  very  high  profit  will  appear  most  representative 
of  that  description,  if  the  description  is  mediocre,  a  mediocre  per¬ 
formance  will  appear  most  representative,  etc.  The  degree  of  favorable¬ 
ness  of  the  description,  of  course,  is  unaffected  by  the  reliability  of 
that  description  or  by  the  degree  to  which  it  permits  accurate  prediction. 
Hence,  if  people  predict  solely  in  terms  of  the  favorableness  of  the  de¬ 
scription,  their  predictions  will  be  insensitive  to  the  reliability  of 
the  evidence  and  to  the  expected  accuracy  of  the  prediction. 

This  mode  of  judgment  violates  the  normative  statistical  theory 
according  to  which  the  extremity  and  range  of  predictions  are  controlled 
by  considerations  of  expected  accuracy.  If  expected  accuracy  is  minimal, 
the  same  predictions  should  be  made  in  all  cases.  Thus,  if  the  descrip¬ 
tions  of  the  various  companies,  for  example,  are  unrelated  to  their  profits, 
the  same  value  (e.g.,  average  profit)  should  be  predicted  for  all  companies. 

If  expected  accuracy  is  perfect,  the  range  of  predicted  values  should  equal 
the  range  of  actual  values.  In  general,  the  greater  the  expected  accuracy, 
the  wider  the  range  of  predicted  values. 

Several  studies  of  numerical  predictions  have  demonstrated  that  intuitive 
predictions  do  not  conform  to  this  rule,  and  that  subjects  show  little  or  no 
regard  for  considerations  of  expected  accuracy  (1).  In  one  of  these  studies, 
subjects  were  presented  with  several  paragraphs,  each  describing  the  perfor¬ 
mance  of  a  student-teacher  during  a  particular  practice  lesson.  Some  subjects 
were  asked  to  evaluate  the  quality  of  the  lesson  described  in  the  paragraph 
in  percentile  scores,  relative  to  a  specified  population.  Other  subjects  were 
asked  to  predict,  also  in  percentile  scores,  the  standing  of  each  of  the 


-11- 


student -teachers  five  years  after  the  practice  lesson.  The  judgments 
made  under  the  two  conditions  were  identical.  That  is,  the  prediction 
of  a  remote  criterion,  (success  of  a  teacher  after  five  years)  ./as 
identical  to  the  evaluations  of  the  information  on  which  the  prediction 
was  based  (the  quality  of  the  practice  lesson).  The  students  who  made 
these  predictions  undoubtedly  knew  that  the  prediction  of  teaching  com¬ 
petence,  on  the  basis  of  a  single  trial  lesson  five  years  earlier,  can 
hardly  be  accurate.  Nevertheless,  their  predictions  were  as  extreme  as 
th*  i r  evaluations. 

5.  The  illusion  of  validity. 

As  we  have  seen,  people  often  predict  by  selecting  the  outcome 
(e.g.,  an  occupation)  that  is  most  representative  of  the  input  (e.g., 
the  description  of  a  person).  The  confidence  they  have  in  their  pre¬ 
diction  depends  primarily  on  the  degree  of  representativeness  attained 
in  the  prediction  (i.e.,  on  the  quality  of  the  match  between  the  selected 
outcome  and  the  input)  with  little  or  no  regard  for  the  factors  that  limit 
predictive  accuracy.  Thus,  people  express  great  confidence  in  the  prediction 
that  a  person  is  a  librarian  when  given  a  description  of  his  personality  which 
matches  the  stereotype  of  librarians,  even  if  the  description  is  scanty,  un¬ 
reliable  or  outdated.  The  unwarranted  confidence  which  is  produced  by  a  good 
between  the  predicted  outcome  and  the  input  information  may  be  called 
the  illusion  of  validity.  This  illusion  persists  even  when  the  judge  is  aware 
of  the  factors  that  limit  the  accuracy  of  his  predictions.  It  is  a  common 
observation  that  psychologists  who  conduct  selection  interviews  often  experience 
considerable  confidence  in  their  predictions,  even  when  they  know  of  the  vast 


literature  that  shows  selection  interviews  to  be  notoriously 
fallible.  The  continued  reliance  on  the  clinical  interview  for 
selection  despite  repeated  demonstrations  of  its  inadequacy  amply 
attests  to  the  strength  of  this  effect. 

Given  input  variables  of  stated  validity,  a  prediction  based 
01  6  variables  can  achieve  higher  accuracy  when  the  input 

variables  are  independent  of  each  other  thar  when  they  are  redundant 
or  correlated.  Redundant  input  variables  generally  yield  input  patterns 
that  appear  internally  consistent,  whereas  uncorrelated  input  variables 
often  yield  input  patterns  that  appear  inconsistent.  The  internal  con¬ 
sistency  of  the  pattern  of  inputs  (e.g.,  a  profile  of  scores)  is  one  of 
the  major  determinants  of  representativeness,  and  hence  of  confidence  in 
a  prediction.  Consequently,  people  tend  to  have  greater  confidence  in  pre¬ 
dictions  based  on  redundant  input  variables  than  in  predictions  based  on 
uncorrelated  variables  (1).  Because  redundancy  among  inputs  usually  increases 
accuracy  and  increases  confidence,  people  tend  to  have  most  confidence  in 
predictions  that  are  very  likely  to  be  off  the  mark. 


6.  Misconceptions  of  Regression. 

Suppose  a  large  group  of  children  have  been  examined  on  two  equi¬ 
valent  versions  of  an  aptitude  test.  If  one  selects  ten  children 
from  among  those  who  did  best  on  one  of  the  two  versions,  he  will  find 
their  performance  on  the  second  version  to  be  somewhat  disappointing, 
on  the  average.  Conversely,  if  one  selects  ten  children  from  among  those 
who  did  worst  on  one  version,  they  will  be  found,  on  the  average,  to  do 
somewhat  better  on  the  other  version.  More  generally,  consider  two  vari¬ 
ables  X  and  Y  which  have  the  same  distribution.  By  and  large,  if  one 
selects  individuals  whose  average  score  deviates  from  the  mean  of  X 
by  k  units,  then  their  average  deviation  from  the  mean  of  Y  will  be  less 
than  k.  These  observations  illustrate  a  general  phenomenon  known  as 

regression  toward  the  mean,  which  was  first  documented  by  Galton  over 
one  hundred  years  ago. 

In  the  normal  course  of  life,  we  encounter  many  instances  of  re¬ 
gression  towards  the  mean,  e.g.,  in  the  comparison  of  the  height  of 
fathers  and  sons,  of  the  intelligence  of  husbands  and  wives,  or  of  the 
performance  of  individuals  on  consecutive  examinations.  Nevertheless, 
p  op  e  do  not  develop  correct  intuitions  about  this  phenomenon.  First, 
they  do  not  expect  regression  in  many  contexts  where  it  is  bound  to 
occur.  Second,  when  they  recognize  the  occurrence  of  regression,  they 
typically  invent  spurious  causal  explanations  for  it  (1).  We  suggest 
that  the  phenomenon  of  regression  remains  elusive  because  it  is  incompa¬ 
tible  with  the  belief  that  the  predicted  outcome  should  be  maximally  re¬ 
presentative  of  the  input,  and  hence  that  the  value  of  the  outcome  variable 
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shou.d  b.  as  creme  as  the  value  of  the  input  variable. 

'it e  tat  lure  to  retegtuze  the  import  of  regression  ca„  have  per. 

nicious  consequences  ic  ^  ,  , 

a  ed  ,,y  the  following  observation.  In 

tl>e  tra‘"ine  °f  Pi,°tS'  «“  execution  of  a  complex  nigh, 

—over  is  liltely  to  he  followed  by  a  deter, oration  on  the  next  attempt, 
-htle  a  poor  Performance  ,s  likely  t„  be  follows  by  an  l>iproren,e„t 
a  standard  manifestation  of  regress, „„  toMrd  ^  TMs  ^  ^ 

cur  even  when  the  instructor  does  not  respond  to  the  trainee's  perfor¬ 
mance.  However,  peonle  tin  nm 

recognize  regression  effects  for  what  they 

ure,  and  invent  unwarranted  n„t.,i 

causa i  explanations  for  them.  Since  flight 

instructors  typically  praise  the  trainee  after  ,  g„„d  performance, 
admonish  him  after  a  poor  performance,  they  tend  to  come  to  the  erroneous 

"',d  POte"tlaUy  t*lat  verbal  rewards  are  detrimental  to 

learning  whereas  punishments  are  beneficial. 

teract ion  as  well  as  in  intentional  training,  rewards 
a.e  typicany  administered  »hen  perforce  is  good  and  punishments  are 

typically  administered  when  n»rf 

P  ormance  is  poor.  By  regression  alone, 

therefore,  behavior  is  most  likeiv  ,  • 

y  to  improve  after  punishment  and  most 

1  ikely  to  deteriorate  after  reward  r 

reward.  Consequently,  the  human  condition  is 

SU;  that’b/  ChM“  •l“1-  -  **  “*  •*«  rewarded  for  phishing  others 

“  ”°St  °fte"  P,,niShed  f°r  "“"“"l  O-pie  are  generaily  not  aware 

°f  this  contingency,  in  fact  t-w0  i 

fact,  the  elusive  role  of  regression  in  determining 

the  apparent  consequences  of  reward 

punishment  seems  to  have  escaped 

the  notice  of  students  of  -his  area. 
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AVA I  LABILITY 

There  ere  situations  in  which  people  assess  the  frequency  of  a 
Hass  or  the  probability  of  an  event  by  the  ease  with  which  instance, 

"F  °CCUrrenCeS  C0U1<I  be  «  “  -*-•  For  example,  one  may  assess 
'•  r.xk  of  heart  attack  among  middle  aged  people  by  recalling  such 

recurrences  among  one's  acquaintances.  Similarly,  one  may  evaluate 

’"'probability  that  a  given  business  venture  will  fail  by  imagining 

various  difficulties  which  it-  ^..1,4 

ies  which  it  could  encounter.  This  judgmental  heuris¬ 
tic  is  called  availability  t«  1 

y.  In  general,  availability  is  a  useful  clue 

for  assessing  frequency  or  probability,  because  instances  of  large 

Classes  arc  recalled  better  and  faster  than  instances  of  less  frequent 

classes.  However,  availability  is  aicn  .  .  . 

y  s  also  affected  by  other  factors  besides 

frequency  and  probability.  Conseouentlv 

y  ^onsequently ,  the  reliance  on  availability 

leads  to  predictable  biases  snm„  *  ..  . 

iases,  some  of  which  are  illustrated  below. 

-?:_Btases_due_to_the  retrievable  re  . . 

When  the  frequency  of  a  class  is  judged  by  the  availability  of  its 
instances,  a  ciass  whose  instances  are  easily  retrieved  will  appear  more 

numerous  than  a  Cass  of  equal  frequency  whose  instances  are  less  retrievable. 
In  an  elementary  demonstration  of  this  effect,  subjects  heard  a  list  of 
well-known  personalities  of  both  sexes  and  were  subsequently  asked  to  judge 
Whether  the  iist  contained  more  names  of  men  than  of  womet.  Different  lists 
were  presented  to  different  groups  of  subjects.  some  of  the  lists  the 

~  "Cre  relatiV'1)'  th‘"  .  -d  in  others  the  women 

were  relatively  more  famous  than  the  men.  In  all  lists  the  s.h'  * 

lists,  the  subjects  errone¬ 
ously  judged  the  classes  consisting  nf  the  mn  e 

g  of  the  more  famous  personalities  to  be 
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the  mor_  numerous  (6) . 

In  addition  to  familiarity,  there  are  other  factors  [e.g., 
salience)  which  affect  the  retrievability  of  instances.  For  example, 
seeing  a  house  burned  down  will  have  a  greater  impact  on  the  subjective 
probability  of  such  accidents  than  merely  reading  about  a  fire  in  the 
local  paper.  Furthermore,  recent  occurrences  are  likely  to  be  relative¬ 
ly  more  available  than  earlier  occurrences.  It  is  a  common  experience 
that  the  subjective  probability  of  an  accident  rises  temporarily  when 
one  sees  a  car  overturned  by  the  side  of  the  road. 


masgs  to  to  the  effectiveness  of  a  search  set. 

Suppose  you  sample  a  word  (of  three  letters  or  more)  at  random  from 
an  fcnglish  text.  Is  it  more  likely  that  the  word  starts  with  r  or  that 
r  is  Us  third  letter?  People  approach  this  problem  by  recalling  words 
that  begin  with  r  (e.g.,  road)  and  words  that  have  r  in  the  third  posi¬ 
tion  (e.g..  car)  and  assess  relative  frequency  by  the  ease  with  which 
words  of  the  two  types  come  to  mind.  Because  it  is  much  easier  to  search 
for  words  by  their  first  than  by  -heir  third  letter,  most  people  judge 
words  that  begin  with  a  given  consonant  to  be  more  numerous  than  words  in 
which  the  same  consonant  appears  in  the  third  position.  They  do  so  even  for 

(O'g-,  r  or  k)  that  are  actually  more  frequent  in  the  third  posi- 
tion  than  in  the  first  (6). 

Different  tasks  elicit  difference  search  sets.  For  example,  suppose  you 
are  asked  to  rate  the  frequency  with  which  abstract  words  (e.g.,  thought,  love) 
concrete  words  (e.g.,  door,  water)  appear  in  written  English.  A  natural 
ay  to  answer  this  question  is  to  search  for  contexts  in  which  the  word  could 
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appear.  It  seems  easier  to  think  of  contexts  in  which  an  abstract 
concept  is  mentioned  (e.g.,  'love'  in  love  stories)  than  to  think 
of  contexts  in  which  a  concrete  word  (e.g.,  'door')  is  mentioned.  If 
the  frequency  of  words  is  judged  by  the  availability  of  the  contexts 
in  which  they  appear,  abstract  words  will  be  judged  as  relatively  more 
numerous  than  concrete  words.  This  bias  has  been  observed  in  a  recent 
>,tudy  (7)  which  showed  that  the  judged  frequency  of  occurrence  of  ab¬ 
stract  words  was  much  higher  than  that  of  concrete  words  of  the  same 
objective  frequency.  Abstract  words  were  also  judged  to  appear  in  a 
much  greater  variety  of  contexts  than  concrete  words. 

9.  Biases  of  imaginabi lity. 

Sometimes,  one  has  to  assess  the  frequency  of  a  class  whose  instances 
are  not  stored  in  memory  but  can  be  generated  according  to  a  given  rule. 

In  such  situations,  one  typically  generates  several  instances,  and  evalu¬ 
ates  frequency  or  probability  by  the  ease  with  which  the  relevant  in¬ 
stances  can  be  constructed.  However,  the  ease  of  constructing  instances 
does  not  always  reflect  their  actual  frequency,  and  this  mode  of  evalua¬ 
tion  is  prone  to  biases.  To  illustrate,  consider  a  group  of  10  people 
who  form  committees  of  k  members,  2  «  k  <  8.  How  many  different  committees 
of  k  members  can  be  formed?  The  correct  answer  to  this  problem  is  given 
by  the  binomial  coefficient \k)  which  reaches  a  maximum  of  252  for  k  »  5. 
Clearly,  the  number  of  committees  of  k  members  equals  the  number  of  committees 
of  (10  -  k)  members  because  any  elected  group  of,  say,  two  members  defines  a 
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unique  non-elected  group  of  8  members. 

A  possible  way  to  answer  this  question  without  computation  is  to 
imagine  several  committees  of  k  members,  and  to  evaluate  the  number  of 

such  committees  by  the  ease  with  which  they  come  to  mind.  Committees 

■!? 

of  few  member?,  sav  2,  are  more  available  than  committees  of  many  mem¬ 
bers,  say  8.  The  simplest  scheme  for  the  construction  of  committees 
is  a  partition  of  the  group  into  disjoint  sets.  One  readily  sees  that  it 
is  easy  to  construct  five  disjoint  committees  of  2  members,  while  it  is 
impossible  to  generate  even  two  disjoint  committees  of  8  members.  Conse¬ 
quently,  if  frequency  is  assessed  by  imaginability,  or  by  availability 
for  construction,  the  small  committees  will  appear  more  numerous  than 
larger  committees,  in  contrast  to  the  correct  symmetric  bell-shaped 
function.  Indeed,  when  naive  subjects  were  asked  to  estimate  the  number 
of  distinct  committees  of  various  sizes,  their  estimates  were  a  decreasing 
monotonic  function  of  the  committee  size  (6).  For  example,  the  median 
estimate  of  the  number  of  committees  of  2  members  was  70,  while  the  esti¬ 
mate  for  committees  of  8  members  was  20  (the  correct  answer  is  45  in  both 
cases) . 

Imaginability  plays  an  important  role  in  the  evaluation  of  probabilities 
in  real-life  situations.  The  risk  involved  in  an  adventurous  expedition,  for 
example,  is  evaluated  by  imagining  contingencies  with  which  the  expedition  is 
not  equipped  to  cope.  If  many  such  difficulties  are  vividly  portrayed,  the 
expedition  can  be  made  to  appear  exceedingly  dangerous,  although  the  ease 
with  which  disasters  are  imagined  need  not  reflect  their  actual  likelihood. 
Conversely,  the  risk  involved  in  an  undertaking  may  be  grossly  underestimated 
if  some  possible  dangers  are  either  difficult  to  conceive,  or  simply  do  not 


come  to  mind. 
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10.  Illusory  correlation 

Chapman  and  Chapman  (8)  have  described  an  interesting  bias  in 
the  judgment  of  the  frequency  with  which  two  events  co-occur.  They 
presented  naive  judges  with  clinical  diagnoses  and  with  test  material 
for  several  hypothetical  patients.  Later  the  subjects  estimated  the 
frequency  with  which  each  diagnosis  (e.g.,  paranoia  or  suspiciousness) 
had  been  accompanied  by  various  symptoms  (e.g.,  peculiarities  in  the 
drawing  of  the  eyes).  The  subjects  markedly  overestimated  the  frequency 
of  co-occurrence  of  natural  associates,  such  as  suspiciousness  and 
peculiar  eyes.  This  effect  was  labeled  illusory  correlation.  In  their 
erroneous  judgments  of  the  data  to  which  they  had  been  exposed,  naive 
subjects  "rediscovered"  much  of  the  common  but  unfounded  clinical  lore 
concerning  the  interpretation  of  the  draw -a-person  test.  The  illusory 
correlation  effect  was  extremely  resistant  to  contradictory  data.  It 
persisted  even  when  the  correlation  between  symptom  and  diagnosis  was 
actually  negative,  and  it  prevented  the  judges  from  detecting  relation¬ 
ships  that  were  in  fact  present. 

Availability  provides  a  natural  account  for  the  illusory- correlation 
effect.  The  judgment  of  how  frequently  two  events  co-occur  could  be  based 
on  the  strength  of  the  associative  bond  between  them.  When  the  association 
is  strong,  one  is  likely  to  conclude  that  the  events  have  been  frequently 
paired.  Consequently,  strong  associates  will  be  judged  to  have  occurred 
frequently  together.  According  to  this  view,  the  illusory  correlation  be¬ 
tween  suspiciousness  and  peculiar  drawing  of  the  eyes,  for  example,  is  due 
to  the  fact  that  suspiciousness  is  more  readily  associated  with  the  eyes 
than  with  any  other  part  of  the  body. 

Life-long  experience  has  taught  us  that,  in  general,  instances  of 
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large  classes  are  recalled  better  and  faster  than  instances  of  less 
frequent  classes;  that  likely  occurrences  are  easier  to  imagine  than 
unlikely  ones:  and  that  the  associative  connections  between  events  are 
strengthened  when  they  frequently  co-occur.  As  a  consequence,  man  has 

t 

at  his  disposal  a  procedure  (i.e.,  the  availability  heuristic)  for 
estimating  the  numerosity  of  a  class,  the  likelihood  of  an  event  ot 
the  frequency  of  co-occurrences,  by  the  ease  with  which  the  relevant 
mental  operations  of  retrieval,  construction,  or  association  can  be 
performed.  However,  as  the  preceding  examples  have  demonstrated,  this 
valuable  estimation  procedure  is  subject  to  systematic  errors. 

ADJUSTMENT  AND  ANCHORING  ** 

In  many  situations,  people  make  estimates  bv  starting  from  an 
initial  value  which  is  adjusted  to  yield  the  final  answer.  The  initial 
value,  or  starting  point,  may  be  suggested  by  the  formulation  of  the 
problem,  or  else  it  may  be  the  result  of  a  partial  computation.  What¬ 
ever  the  source  of  the  initial  value,  adjustments  are  typically  in¬ 
sufficient  (4).  That  is,  different  starting  points  yield  different 
estimates,  which  are  biased  towards  the  initial  values.  We  call  this 
phenomenon  anchoring. 

11.  Insufficient  adjustment. 

In  a  demonstration  of  the  anchoring  effect  people  were  asked  to 
estimate  various  quantities,  stated  in  percentages  (e.g.,  the  percentage 
of  African  countries  in  the  U.N.).  For  each  question  a  starting  value 
between  0  and  100  was  determined  by  spinning  a  wheel  of  fortune  in  the 
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subjects'  presence.  The  subjects  were  instructed  to  indicate  whether 
the  given  (arbitrary)  starting  value  was  too  high  or  too  low,  and  then 
to  reach  their  estimate  by  moving  upward  or  downward  from  that  value. 
Different  groups  were  given  different  starting  values  for  each  problem. 

These  arbitrary  values  bad  a  marked  effect  on  the  estimates.  For  example, 
the  median  estimates  of  the  percentage  of  African  countries  in  the  U.N. 

were  25*  and  45%,  i .  spec! j  ve!y,  for  groups  which  received  10%  and  65%  as 
starting  points.  Payoff  for  accuracy  did  not  reduce  the  anchoring  effect.. 

Anchoring  occurs  nor.  only  when  the  starting  point  is  given  to  the 
subject  but  also  when  the  subject  bases  his  estimate  on  the  result  of  some 
incomplete  computation.  A  study  of  intuitive  numerical  estimation  illustrates 
this  effect.  Iwo  groups  of  high.srhool  students  estimated,  within  5  seconds, 
a  numerical  expression  that  was  written  on  the  blackboard.  One  group  esti¬ 
mated  the  product  8  x  7  x  6  x  5  x  4  x  3  x  2  x  1,  while  another  group  esti¬ 
mated  the  product  Ix2x3x4x5x  0x7x8.  To  rapidly  answer  such 
questions  people  perform  a  few  steps  of  computation  and  estimate  the  pro¬ 
duct  by  extrapolation  or  adjustment.  Because  adjustments  are  typically 
insufficient,  this  procedure  should  lead  to  underestimation.  Furthermore, 
because  the  result  ot  the  first  few  steps  of  multiplication  (performed  from 
left  to  right)  is  higher  in  the  descending  sequence  than  in  the  ascending 
sequence,  the  former  expression  should  be  judged  larger  than  the  latter. 

Both  predictions  were  confirmed.  The  median  estimate  for  the  ascending 
sequence  was  512,  while  the  median  estimate  for  the  descending  sequence 
was  2,250.  The  correct  answer  is  40,320. 
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— -  Biases  in  the  evaluation  of  conjunctive  and  disjunctive  event!. 

In  a  recent  study  (9),  subjects  were  given  the  opportunity  to  bet 
on  one  of  two  events.  Three  types  of  events  were  used:  (i)  simple  events, 
e.g.,  drawing  a  red  marble  from  a  bag  containing  50%  red  marbles  and  50% 
white  marbles,  (ii)  conjunctive  events,  e.g.,  drawing  a  red  marble  7  times 
m  succession,  with  replacement,  from  a  bag  containing  90%  red  marbles 
and  10%  white  marbles,  (iii)  disjunctive  events,  e.g.,  drawing  a  red  marble 
at  least  once  in  7  successive  tries,with  replacement,  from  a  bag  containing 
10%  red  marbles  and  90%  white  marbles.  In  this  problem,  a  significant 
majority  of  subjects  preferred  to  bet  on  the  conjunctive  event  (the  pro¬ 
bability  of  which  is  .48}  rather  than  on  the  simple  event,  the  probability 
of  which  is  .50.  Subjects  also  preferred  to  bet  on  the  simple  event  rather 
than  on  the  disjunctive  event  which  has  a  probability  of  .52.  Thus,  most 
subjects  bet  on  the  less  likely  event  in  both  comparisons,  '.ms  pattern 
of  choices  illustrates  a  general  finding.  Studies  of  choice  among  gambles 
and  of  judgments  of  probability  indicate  that  people  tend  to  overestimate 
the  probability  of  conjunctive  events  (10)  and  to  underestimate  the  pro¬ 
bability  of  disjunctive  events.  These  biases  are  readily  explained  as 
effects  of  anchoring.  The  stated  probability  of  the  elementary  event  (e.g., 
of  success  at  any  one  stage)  provides  a  natural  starting  point  for  the  esti¬ 
mation  of  the  probabilities  of  both  conjunctive  and  disjunctive  events.  Since 
adjustment  from  the  starting  point  is  typically  insufficient,  the  final  esti¬ 
mates  remain  too  close  to  the  probabilities  of  the  elementary  events  in  both 
cases.  Note  that  the  overall  probability  of  a  conjunctive  event  is  lower  than 
the  probability  of  eac,  elementary  event,  whereas  the  overall  probability  of 
a  disjunctive  event  is  higher  than  the  probability  of  each  elementary  event. 
As  a  oonsequence  of  anchoring,  the  overall  probability  will  be  overestimated 


in  conjunctive  problems  and  underestimated  in  disjunctive  problems. 

Biases  in  the  evaluation  of  compound  events  are  particularly 
significant  in  the  context  of  planning.  The  succescful  completion  of 
an  undertaking  (e.g.,  the  development  of  a  new  product)  typically  has 
9  conjunctive  character:  for  the  undertaking  to  succeed  each  of  a  series 
of  events  must  occur.  Even  when  each  of  these  events  is  very  likely,  the 
overall  probability  of  success  can  be  quite  low  when  the  number  of  events 
is  large.  The  gf ne  a t  tendency  to  overestimate  the  probability  of  conjunc¬ 
tive  events  leads  to  unwarranted  optimism  in  the  evaluation  of  the  like¬ 
lihood  that  a  plan  will  succeed,  or  that  a  project  will  be  completed  on 
time.  (diversely,  disjunctive  structures  are  typically  encountered  in  the 
evaluation  of  risks.  A  complex  system  (e.g.,  a  nuclear  reactor  or  a  human 
body)  will  malfunction  if  any  of  its  essential  components  fails.  Even 
when  the  likelihood  of  failure  in  each  component  is  slight,  the  probability 
of  an  overall  failure  can  be  high  if  many  components  are  involved.  Because 
of  anchoring,  people  wi 1 1  tend  to  underestimate  the  probabilities  of  failure 
in  complex  systems.  Thus,  the  direction  of  the  anchoring  bias  can  sometimes 
be  inferred  from  the  structure  of  the  event.  The  chain- like  structure  of 

conjunctions  leads  to  overestimation,  the  funnel -like  structure  of  disjunctions 
leads  to  underestimation. 

Anchoring  in  the  assessment  of  subject! ve  probability  distributions. 

For  many  purposes  (e.g.,  the  calculation  of  posterior  probabilities, 
decision-theoretical  analyses)  a  person  is  required  to  express  his  beliefs 
about  a  quantity  (e.g.,  the  value  of  the  Dow-Jones  on  a  particular  day)  in 
the  form  of  a  probability  distribution.  Such  a  distribution  is  usually  con- 
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structed  by  asking  the  person  to  seleet  values  of  the  quantity  that 
correspond  to  specified  percentiles  of  his  subjective  probability  dis¬ 
tribution.  Por  example,  the  judge  nay  be  asked  to  select  a  number  X 

QO 

such  that  his  subjective  probability  that  this  number  will  be  higher 
than  the  value  of  the  Dow-Jones  is  .90.  That  is.  he  should  select  X 
SO  that  he  is  just  willing  to  accept  9  to  1  odds  that  the  Dow-Jones 

will  not  exceed  V  A  subjective  probability  distribution  for  the 

value  of  the  Dow-Jones  can  he  construct  r 

constiucted  from  several  such  judgments 

corresponding  to  different  percentiles  (e.g  X  X  x  v 

g  >  *10,  X25,  X?5,  X99,  etc.) 

By  collecting  subjective  probability  distributions  for  many  different 

quantities,  it  is  possible  to  test  the  judge  for  proper  calibration.  A 

judge  is  properly  (or  externally,  calibrated  in  a  sot  of  problem  if 

exactly  Ilk  of  the  true  vaiues  of  the  assessed  quantities  fail  below  his 

stated  values  ef  X„.  For  example,  the  tiue  values  should  fall  below  x 

for  1%  of  the  quantities  and  above  X  for  1%  th  01 

a99  0r  n  of  the  quantities.  Thus, 

the  true  values  should  full  in  the  . 

n  the  confidence  interval  between  X  and  X 

on  98%  of  the  problems. 

Several  investigators  (e.g..  a.  12,  13,  haye  obtalned  probabmty  ^ 

butions  for  nany  quantities  fro.  a  Urge  number  of  judges.  These  distribu- 

tions  indicated  large  and  cvct- •  . 

8  and  systematic  departures  from  proper  calibration. 

u  most  studies,  the  actuai  values  of  the  assessed  quantities  are  either 

smaller  than  XQ1  ur  greater  than  Xgg  f„,  about  301  of  the  problems.  That 

is,  the  subjects  state  overly  narrow 

y  rrow  confidence  intervals  which  reflect 

more  certainty  than  is  justified  •  . 

justified  by  their  knowledge  about  the  assessed 

quantities.  This  bias  is  shared  by  na<ve  as  well  ..  a, 

1  n  _ve  as  well  as  sophisticated  subjects. 
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and  i*-  is  not  eliminated  by  introducing  proper  scoring  rules  which 
provide  incentives  for  external  calibration.  This  effect  is  readily 
interpreted  as  an  instance  of  anchoring.  To  select  XgQ  for  the  value 
of  the  Dow-Jones,  for  example,  it  is  natural  to  begin  by  thinking  about 
one's  best  estimate  of  the  Dow-Jones  and  to  adjust  this  value  upward.  If 
this  adjustment  -  like  most  others  -  is  insufficient,  then  XgQ  will  not 
be  sufficiently  extreme.  A  similar  anchoring  effect  will  occur  in  the 
selection  of  XJ()  which  is  presumably  obtained  by  adjusting  one's  best 
estimate  downwards.  Consequently,  the  confidence  interval  between  X1Q 
and  Xg(>  will  be  too  narrow,  and  the  assessed  probability  distribution 
will  be  too  tight.  in  support  of  this  interpretation  it  can  be  shown 
that  the  tightness  of  subjective  probability  distribution,  is  eliminated 
by  a  procedure  in  which  one's  best  estimate  does  not  serve  as  an  anchor. 

Subjective  probability  distributions  for  a  given  quantity  (e.g.,  the 
Dow-Jones)  can  be  obtained  in  two  different  ways.  (a)  By  asking  the  subject 
to  select  values  for  the  Dow-Jones  that  correspond  to  specified  percentiles 
of  his  probability  distribution,  (h)  By  asking  the  subject  to  assess  the 
probability  that  the  true  value  of  the  Dow-Jones  will  exceed  some  specified 
values.  The  two  procedures  are  formally  equivalent  and  should  yield  identi¬ 
cal  distributions.  However,  they  suggest  different  modes  of  adjustment  from 
different  anchors.  In  procedure  (a) ,  the  judge  states  his  answer  in  units  of 
the  assessed  quantity,  and  the  natural  starting  point  is  his  best  estimate. 
In  procedure  (b)  the  answers  are  stated  in  odds  or  probabilities  and  the 
natural  starting  point  is  even  odds  or  a  probability  of  one-half.  Anchoring 
on  the  starting  point  in  procedure  (b)  will  yield  conservative  estimates  of 
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odds,  i.e.i  odds  that  are  too  close  to  1:1  and  probability  distributions 
that  are  too  flat. 

To  contrast  the  two  procedures,  a  set  of  24  quantities  (e.g.,  the 
air  distance  New  Delhi-Peking)  was  presented  to  one  group  of  subjects 
who  assessed  either  Xj^r  for  each  problem.  Another  group  of  subjects 
received  the  median  judgment  of  the  first  group  for  each  of  the  24  quan¬ 
tities.  They  were  asked  to  assess  the  odds  that  each  of  the  given  values 
exceeded  the  true  value  of  the  relevant  quantity.  In  the  absence  of  any 
bias,  the  subjects  in  the  second  group  should  retrieve  the  odds  specified 
to  the  first:  group,  i.e.,  9:1.  If  the  subjects  in  the  second  group  are 
anchored  on  even  odds,  however,  their  stated  odds  should  be  less  extreme, 
i.e.,  closer  to  1:1.  Indeed,  the  median  odds  stated  by  this  group,  across 
all  problems,  were  3:1.  When  the  judgments  of  the  two  groups  were  tested 

for  external  calibration  t  it  was  founcj  that  the  judgments  of  the  first  group 
were  indicative  of  overly  tight  probability  distributions,  in  accord  with 

earlier  results,  whereas  the  odds  stated  by  the  second  group  were  indicative 
of  overly  flat  probability  distributions.  This  observation  suggests  the 
intriguing  possibility  that  an  appropriate  combination  of  the  two  methods 
could  yield  properly  calibrated  probability  distributions. 


DISCUSSION 

The  preceding  sections  described  some  heuristics  that  are  commonly 
employed  in  judgments  about  uncertain  events,  and  demonstrated  several 
biases  to  which  these  judgments  are  susceptible.  In  the  present  section 
we  discuss  the  nature  of  these  heuristics  and  biases  and  their  place  in 
the  analysis  of  rational  judgment. 


U" 
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The  biases  with  which  we  are  concerned,  like  perceptual  errors 
and  illusions,  are  characteristic  of  the  coenitive  operations  by  which 
impressions  and  judgments  are  formed.  These  cognitive  biases  ere  distinct 
rom  the  better-known  intrusions  of  emotional  and  motivational  factors  into 
judgment,  such  as  wishful  thinking  and  the  intentional  distortions  tf  judg¬ 
ment  induced  by  payoffs  and  penalties.  The  biases  described  in  this  paper 
are  consequences  of  the  reliance  on  heuristics  such  as  representativeness 
and  availability,  and  they  are  not  attributable  to  motivational  consideration,. 
Indeed,  several  of  the  severe  errors  of  judgments  reported  earlier  were 
Observed  despite  the  fact  that  subjects  were  encouraged  to  be  accurate  and 
were  rewarded  for  the  correct  answers.  For  example.  the  common  erroneous 
belief  that  there  are  more  words  in  an  English  text  that  begin  with  r  than 
which  r  is  the  third  letter  was  not  shaken  by  monetary  payoffs  for 
accuracy  (6).  Similarly,  offering  the  subjects  a  $1  bonus  for  the  correct 
answer  did  not  increase  the  prevalence  of  the  belief  that  a  daily  list  of 

births  in  which  more  than  60%  of  the  babies  are  boys  is  more  likely  in  a 
small  hospital  than  in  •>  large  one  (2). 

The  reliance  on  heuristics  and  the  presence  of  common  biases  are 
general  characteristics  of  intuitive  judgments  under  uncertainty. 

They  apply  not  only  to  laymen  untutored  in  the  laws  of  probability, 
but  also  to  experts  -  when  they  think  intuitively.  For  example,  the 
tendency  to  predict  the  outcome  that  best  represents  the  individuating 
■  information,  with  insufficient  regard  for  prior  probability,  has  been  observed 
in  the  intuitive  judgments  of  individuals  who  had  extensive  training  in  sta¬ 
tistics  (1).  Although  some  common  errors  (e.g.,  the  gambler's  fallacy)  are 
easily  avoided  by  the  statistically  sophisticated,  there  is  evidenee  that 
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the  intuitive  judgments  of  experts  are  prone  to  similar  errors  in  more 
intricate  and  less  familiar  questions  (e.g.,  the  birthday  problem). 

It  is  not  surprising  that  heuristics  such  as  representativeness  and 
availability  are  not  discarded,  even  though  they  occasionally  lead  to 
errors  in  prediction  or  estimation.  What  is  perhaps  surprising  is  the 
failure  or  peop  e  to  infer  from  life- long  experience  such  fundamental 
statistical  rules  as  regression  towards  the  mean,  or  the  effect  of  sample 
size  on  sampling  variability.  Although  everyone  is  exposed  in  the  normal 
course  of  life  to  numerous  examples  from  which  these  rules  could  have  been 
induced,  very  few  people  discover  the  principles  of  sampling  and  regression 
on  their  own.  The  main  cause  for  the  failure  to  develop  valid  statistical 
intuitions  is  that  events  are  normally  not  coded  in  terms  of  the  features 
that  are  crucial  to  the  learning  of  statistical  rules.  Although  we  encounter 
many  samples  of  different  sizes  from  the  same  population  (e.g.,  lines,  para¬ 
graphs  and  pages  m  texts)  we  rarely  compare  the  statistical  properties  of 
such  samples,  e.g.,  their  average  word  length.  Consequently, 
we  do  not  have  an  effective  opportunity  to  discover  that,  in  general, 
successive  pages  differ  less  in  average  word  length  than  do  successive  lines. 
People  just  do  not  think  about  texts  in  this  manner. 

When  events  are  coded  into  natural  categories,  the  probabilities  or 
relative  frequencies  of  these  categories  are  learned  without  difficulty.  It 
is  the  lack  of  an  appropriate  code  that  explains  why  people  usually  do  not 
detect  the  biases  in  their  own  judgments.  A  person  could  conceivably  learn 
whether  his  probability  judgments  are  externally  calibrated  by  keeping  a  tally 
of  the  proportion  of  events  that  actually  occur  among  those  to  which  he  assigns 
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the  same  probability.  However,  it  is  not  natural  to  group  events  by 
their  judged  probability.  In  the  absence  of  appropriate  grouping  of 
events,  the  only  available  feedback  is  whether  individual  events  did 
or  did  not  occur.  This  dichotomous  feedback  provides  little  information 
concerning  the  adequacy  of  one's  judgments  of  probability.  Thus,  the 
failure  to  realize  that  judgmental  operations  are  repetitive  -  even 
when  they  apply  to  unique  events  -  is  a  major  obstacle  for  effective 
learning. 

Modern  decision  theory  (14,15)  regards  subjective  probability  as 
the  quantified  opinion  of  an  idealized  person.  Specifically,  the  sub¬ 
jective  probability  of  a  given  event  is  defined  by  the  set  of  bets  about 
this  event  which  such  a  person  is  willing  to  accept.  An  internally  con¬ 
sistent,  or  coherent,  subjective  probability  measure  can  be  derived  for 
an  individual  if  his  choices  among  bets  satisfy  certain  principles  (i.e., 
the  axioms  of  the  theory).  The  derived  probability  is  subjective  in  the 
sense  that  different  individuals  are  allowed  to  nave  different  probabi¬ 
lities  for  the  same  event.  Naive  or  intuitive  judgments  of  probability 
typically  fail  to  satisfy  the  necessary  axioms.  The  theory  of  subjective 
probability  provides  a  rationale  for  a  procedure  in  which  estimates  are 
modified  or  corrected  to  achieve  internal  consistency.  The  inherently 
subjective  nature  of  probability  judgments  has  led  many  writers  to  the 
belief  that  internal  consistency  is  the  only  criterion  by  which  judged 
probabilities  should  be  evaluated.  From  the  standpoint  of  the  formal 
theory  of  subjective  probability,  any  set  of  internally  consistent  pro¬ 
bability  judgments  is  as  good  as  any  other. 
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Our  position  is  that  internal  consistency  alone  does  not  guarantee 
the  adequacy  of  a  set  of  probability  judgments  because  an  internally 
consistent  set  of  subjective  probabilities  can  be  incompatible  with 
other  beliefs  held  by  the  individual.  Consider  a  person  whose  subjec¬ 
tive  probabilities  for  all  possible  outcomes  of  a  coin-tossing  game 
reflect  the  gambler's  fallacy.  That  is,  his  estimate  of  the  probability 
tails  0,1  »ny  toss  increases  with  the  number  of  consecutive  heads  that 
preceded  that  toss.  "Hie  judgments  of  such  a  pe“*>on  could  be  internally 
consistent  and  therefore  acceptable  as  adequate  subjective  probabilities 
according  to  the  criterion  of  the  formal  theory.  These  probabilities, 
however,  are  incompatible  with  the  generally-held  belief  that  a  coin  has 
no  memory  and  is  therefore  incapable  of  generating  sequential  dependencies. 

For  judged  probabilities  to  be  considered  adequate,  or  rational,  inter¬ 
nal  consistency  is  not  enough.  The  judgments  must  be  compatible  with  the 
entire  web  of  beliefs  held  by  the  individual.  Compatibility  among  beliefs 
is  the  essence  of  rational  judgment.  This  criterion  is  more  stringent  than 
internal  consistency  but  also  more  appropriate  because  it  requires  that  a 
set  of  judgments  be  compatible  with  the  judge's  entire  body  of  knowledge 
and  not  only  consistent  within  itself.  Unfortunately,  there  can  be  no  simple 
formal  procedure  for  assessing  the  compatibility  of  a  set  of  probability 
judgments  with  the  judge's  total  system  of  beliefs.  Nevertheless,  the  rational 
judge  will  strive  for  compatibility,  even  though  internal  consistency  is  more 
easily  achieved  and  assessed.  In  particular,  he  will  attempt  to  make  his 
probability  judgments  compatible  with  his  knowledge  about  (i)  the  subject- 
matter;  (ii)  the  laws  of  probability;  (iii)  his  own  judgmental  heuristics 
and  biases.  The  present  view  provides  a  rationale  for  a  procedure  in  which 
judged  probabilities  are  modified  or  corrected  to  achieve  a  higher  degree 
of  compatibility  with  all  these  types  of  knowledge. 
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We  began  this  paper  with  the  question  of  how  people  make 
intuitive  judgments  of  probability.  The  answers  appears  to  be  that 
such  judgments  are  based  on  the  outcomes  of  some  specified  mental 
operations  such  as  the  assessment  of  representativeness  or  availa¬ 
bility.  In  the  formal  theory  of  subjective  probability,  the  mental 
operation  performed  by  the  idealized  judge  is  a  choice  between  bets. 

Although  subjective  probabilities  can  sometimes  be  inferred  from 
choices  between  bets,  people  normally  do  not  evaluate  probabilities 
in  this  manner.  In  fact,  judgments  of  likelihood  usually  determine 
preferences  among  bets  and  are  not  derived  from  them,  as  in  the 
axiomatic  theory  of  subjective  probability  (M*  IS). 

SUMMARY 

This  paper  describes  three  heuristics,  or  mental  operations,  that 
are  employed  in  judgment  under  uncertainty,  (i)  An  assessment  of 
representativeness  or  similarity,  which  is  usually  performed  when  people 
are  asked  to  judge  the  likelihood  that  an  object  or  event  A  belong*  to  a 
class  or  process  B.  (ii)  An  assessment  of  the  availability  of  instances 
or  scenarios,  which  is  often  employed  when  people  are  asked  to  assess  the 
frequency  of  a  class  or  the  plausibility  of  a  particular  development,  (iii) 

An  adjustment  from  a  starting  point,  which  is  usually  employed  in  numerical 
prediction  when  a  relevant  value  is  available.  These  heuristics  are  highly 
economical  and  usually  effective,  but  they  lead  to  systematic  and  predictable 
errors.  A  better  understanding  of  these  heuristics  and  of  the  blase*  to 
which  they  lead  could  improve  judgments  and  decisions  in  situations  of 
uncertainty. 
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